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© Motion vector processing in digital television images. 



CM 



© A television standards converter comprises a 
motion analyzer (36 to 40) for analyzing the motion 
between consecutive fields of an input television 
signal of one television standard and for deriving 
motion vectors in dependence on the motion, a filter 
(47) for recursively filtering the motion vectors, and 
an interpolator (1) for aligning the fields in depen- 
dence on the filtered motion vectors so as effectively 
to represent static pictures, and means (1) to effect 
conversion using the static pictures to derive the 
required output television signal of a different televi- 
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MOTION VECTOR PROCESSING IN DIGITAL TELEVISION IMAGES 



This invention relates to motion vector process- 
ing in digital television images. Such processing is 
used in television standards converters and in slow 
motion processors. 

International television programme exchange 5 
necessitates standards converters due to the dif- 
ferent television standards used in different coun- 
tries, for example, the 625-line 50-fieids per second 
(625/50) PAL system used in the UK. and the 525- 
line 60-fields per second (5256Q) NTSC system w 
used in the USA. 

Many different standards converters have been 
previously proposed. One of the best known is the 
ACE (Advanced Conversion Equipment) developed 
by the British Broadcasting Corporation. Basically 75 
ACE operates on an input digital television signal 
line-by-line to derive interpolated samples required 
to form an output digital television signal. Interpola- 
tion is done not only spatially using four successive 
horizontal scan lines of the input television signal, 20 
but also temporally using four successive fields of 
the input television signal. Thus, each line of the 
output television signal is derived by multiplying 
respective samples from sixteen lines of the input 
television signal by respective weighting coeffi- 25 
cients. 

Further details of ACE will be found in UK 
patent specification GB-A-2 059 712 and in 'Four- 
field digital standards converter for the eighties' by 
R N Robinson and G J Cooper at Pages 11 to 13 
^f^el§vis16nW(th^jpM 
Society) for January/February 1982. 

Although ACE gives good results, there is the 
problem that the equipment is very bulky. To over- 
come this problem, we have previously proposed a 
television standards converter comprising three 
field stores and four 4-line stores for receiving an 
input digital television signal of one standard and 
deriving therefrom arrays of sixteen lines, each 
array consisting of four successive lines from each 40 
of four successive fields of the input television 
signal. A weighting coefficient store stores sets of 
sixteen weighting coefficients, respective sets cor- 
responding to positions both spatial and temporal 
of respective lines of an output digital television 45 
signal of a different standard, relative to the sixteen 
lines of the input television signal. Two interpolation 
filters then derive line by-line the output television 
signal by multiplying corresponding sample values 
from each of the sixteen lines of the input television so 
signal by a respective weighting coefficient in a set 
of weighting coefficients and sum the resulting 
products to form an interpolated sample value, and 
four output field stores receive and store the de- 
rived lines of the output television signal. To store 



30 



35 



the additional lines which are derived when the 
output television signal has more lines than the 
input television signal, a 45-line store is interposed 
between one of the interpolation filters and the 
output field stores. Further details will be found in 
our UK patent specification GB-A-2 140 644. 

The performance of such standards converters 
which employ vertical/temporal interpolation tech- 
niques represents a compromise between generat- 
ing blurred pictures while maintaining good motion 
portrayal and maintaining vertical resolution but at 
the expense of 'judder 1 . The former is a result of 
post filtering in order to prevent disturbing alias 
effects; the latter is a result of the intrusion of the 
adjacent two-dimensional repeat sample structures. 

We have therefore proposed that motion vector 
estimation should be incorporated in television 
standards converters and in slow motion proces- 
sors. The problem with the majority of existing 
motion vector estimation methods is that their use 
is biased towards video conference type applica- 
tions where generally the subject matter is either a 
single person's head and shoulders or a small 
group of people seated around a table. With televi- 
sion images of this type the motion is relatively 
simple in comparison with broadcast television im- 
ages where for example at a horse race meeting 
the camera could be following the leaders in a 
race. In this situation the motion would be complex, 
for example, because the camera would be pan- 

speeds greater than eight pixels per field, while in 
the foreground there would be at least one horse 
galloping. This means that the motion vector es- 
timation method must try to track the horses tegs, 
which may well be moving in different directions to 
that of the already moving background. 

There are therefore problems in estimating ap- 
propriate motion vectors and in particular in arriving 
at a final single motion vector corresponding to 
each pixel of the television image, such that the 
motion of that pixel can be represented smoothly 
even in the presence of discontinuities such as 
scene changes, and some noise. 

According to the present invention there is 
provided a television standards converter compris- 
ing: 

means for analyzing the motion between consecu- 
tive fields of an input television signal of one televi- 
sion standard and for deriving motion vectors in 
dependence on said motion; 
characterized by: 

means recursively to filter said motion vectors; 
means then to align said fields in dependence on 
said filtered motion vectors so as effectively to 
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represent static pictures; and 
means to effect conversion using said static pic- 
tures to derive the required output television signal 
of a different television standard. 

According to the present invention there is also 
provided a 625-line 50-fields per second to 525-line 
60- fie Ids per second television standards converter 
comprising: 

a first time base corrector for receiving an input 
625-line 50-fields per second digital television sig- 
nal; 

characterized by: 

a motion analyzer connected to the output of said 
time base corrector for analyzing motion in said 
input television signal and for deriving motion vec- 
tors in dependence on said motion; 
means recursively to filter said motion vectors; 
a shift register also connected to the output of said 
first time base corrector; ; 
an interpolator for deriving samples of a required 
output 525-line 60-fields per second digital televi- 
sion signal in dependence on samples derived 
from said shift register and said filtered motion 
vectors; and a second time base corrector for as- 
sembling said derived samples to form said output 
television signal. 

According to the present invention there is also 
provided a 525-line 60-fields per second to 625-line 
50-fields per second television standards converter 
comprising: 

a first time base converter for receiving an input 
525-line 60-fieids per second digital tetevision sig- 
nal; 

characterized by: 

a motion analyzer connected to the output of said 
time base corrector for analyzing motion in said 
input television signal and for deriving motion vec- 
tors in dependence on said motion vectors; 
means recursively to filter said motion vectors; 
a shift register also connected to the output of said 
first time base corrector; 

an interpolator for deriving samples of a required 
output 625-line 50 fields per second digital televi- 
sion signal in dependence on samples derived 
from said shift register and said filtered motion 
vectors; and a second time base corrector for as- 
sembling said derived samples to form said output 
television signal. 

According to the present invention there is also 
provided a slow motion processor comprising: 
an input circuit for receiving an input digital televi- 
sion signal; 

a motion analyzer for analyzing motion in said input 
digital television signal and for deriving motion vec- 
tors in dependence on said motion; 
characterized by: 

means recursively to filter said motion vectors; 

a shift register for holding successive different 



fields of said input television signal; 
an interpolator for deriving samples of a required 
slow motion output digital television signal in de- 
pendence on the degree of slow motion, samples 
5 derived from said shift register, and said filtered 
motion vectors; and 

a 2-field time base corrector for assembling said 
derived samples to form said slow motion output 
television signal. 

jo The invention will now be described by way of 
example with reference to the accompanying draw- 
ings, throughout which like parts are referred to by 
like references, and in which; 

Figure 1 shows in very simplified block dia- 

;s grammatic form a television standards converter; 

Figure 2 shows in block diagrammatic form a 
first television standards converter; 

Figure 3 shows in block diagrammatic form a 
second television standards converter; 

20 Figure 4 shows part of ihe standards con- 

verter of Figure 3 in more detailed block diagram- 
matic form; 

Figure 5 shows time charts for explaining the 
operation of Figure 4; 
25 Figure 6 shows part of the standards con- 

verter of Figure 2 in more detailed block diagram- 
matic form; 

Figure 7 shows time charts for explaining the 
operation of Figure 6; 
30 Figure 8 shows part of the standards con- 

verter of Figure 2 in more detailed block diagram- 
matic form; 

Figure 9 shows time charts for explaining the 
operation of Figure 8; 
35 v Figure 10 shows part of the standards con- 
verter of Figure 2 in more detailed block diagram- 
matic form; 

Figure 11 A shows diagrammatically the mo- 
tion of an object in sequential fields; 
40 Figure 11 B shows corresponding motion 

vectors; 

Figure 12 shows in block form the compari- 
son of motion vectors; 

Figures 13 to 15 show respective examples 
4$ of motion vectors from three sequential fields; 

Figure 1 6 shows the effect of recursive filter- 
ing of motion vectors; 

Figure 17 shows in block form a general 
example of an infinite impulse response filter; 
so Figure 18 shows diagrammatically negative 

vector filtering; 

Figure 19 shows in block form an infinite 
impulse response filter for negative vector filtering; 
Figure 20 shows diagrammatically positive 
55 vector filtering; and 

Figure 21 shows in block form an infinite 
impulse response filter for positive vector filtering. 
In order more readily to understand the motion 



3 



EP 0 294 957 A2 



vector processing to be described, the form and 
operation of two standards converters and a slow 
motion processor in accordance with the present 
invention which use such motion vector processing 
will first be described. The standards converters to 5 
be described maintain vertical resolution and re- 
move the 'judder' by compensating for motion be- 
tween fields. In effect the motion between consecu- 
tive fields is analyzed. These fields can then be 
'aligned 1 pixel by pixel such that they represent io 
static pictures upon which conversion can then take 
place. As a result, vertical resolution can be main- 
tained. 

The standards converters to be described can 
be divided into two parts. The first part is analo- 75 
gous to a known standards converter performing 
vertical/temporal interpolation to convert between 
525/60 and 625/50 television standards. Alone, this 
would generate an output in which vertical resolu- 
tion would be maintained but with the added effect 20 
of judder. To remove this judder four fields of the 
input digital television signal which are used in the 
conversion process are aligned under the control of 
motion vectors generated from a motion analyzer 
which forms the second part of the standards con- 25 
verter. 

This is shown in very simplified diagrammatic 
block form in Figure t. The video portion of an 
input digital television signal of one standard, which 
* may for example have been derived by sampling 
an analogue television signal at 13.5 MHz, is sup- 
plied to an interpolator 1 from which the video 
portion of the required output television signal of a 
different standard is derived. A motion analyzer 2 
receives the nam inmc©^iaeo w aVia^eri ves^ofi^ 
vectors which provide data representing the motion 
between successive fields of the input television 
signal to control the operation of the interpolator 1. 
The interpolator 1 operates in a generally similar 
manner to the corresponding portion of a known aq 
standards converter, for example as referred to 
above. It also, however, contains the means to 
align the four fields used in the interpolation, under 
the control of the motion vectors. 

The repositioning of the four fields is per- 
formed in two stages. The first stage involves vary- 
ing the address of a variable delay element asso- 
ciated with each field to reposition the picture to 
the nearest line or sample. The second stage uses 
interpolation techniques both vertically and horizon- 
tally to reposition to within ±1/16 line or ±1/8 of a 
sample. Even with no movement, both the above 
techniques are used to enable conversion of line 
standards. 

The vertical interpolator has four taps per field 55 
allowing effectively an 8-tap vertical filter to be 
applied to the static pictures. An 8-tap interpolator 
allows good vertical resolution to be maintained 
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with minimal distortion. The effect of distortion in 
the horizontal interpolator is iess of a problem, so a 
2-tap horizontal filter is used, although a 4-tap 
horizontal filter, for example, may be used. 

The temporal interpolator is used in normal 
operation to enable interpolation of perspective 
changes or when no sensible motion vector can be 
detected, in which case the interpolator 1 must 
revert to normal standards conversion operation 
where no picture re-positioning occurs. 

When converting from a high field rate to a 
lower rate, the incoming fields are interpolated 
such that an interpolated field can occasionally be 
dropped without any movement deterioration. All 
the interpolation is done at the input field rate and 
passed to a time base corrector which then 
spreads the fields generated over the required time 
period for the output standard. 

The above operation is necessary when con- 
verting from 525/60 to 625/50. It is also evident 
however that 625 lines must be generated where 
only 525 lines exist in the input signal. 

To overcome the line number conversion prob- 
lem a second time base corrector is used at the 
input to produce a signal having 585 lines at the 60 
Hz rate. A 585-line format can contain all the active 
picture information in the 625-line format. Following 
this first time base corrector there are occasional 
lines which have no video information. The inter- 
polator stores are frozen during this time, so that 
an additional interpolated line can be generated 
from the same lines used to generate the previous 
output line. This process allows 625 lines to be 
interpolated from the original 525. 

inhe^rease^ 

now be explained in more detail. A 625-line picture 
contains 288 active lines in each field, and 720 
samples in each" horizontal line at the sampling rate 
of 13.5 MHz. The circuits, to be described below, 
of the television standards converters of Figure 2 
and 3 use techniques which allow the picture to be 
shifted horizontally by plus or minus twenty-four 
samples. This requires a minimum horizontal blan- 
king of forty-eight samples. The total number of 
sample positions required in a field is therefore: 

(720 + 48) x 288 = 221184. 
There are clearly considerable advantages in using 
a 13.5 MHz clock throughout the system, in which 
case the number of clock cycles within a 60 Hz 
period (more exactly a 59.94 Hz period) is: 
225225. 

If 576 lines of data are required in one frame, 
the number of horizontal samples would be 
782.03125. Although this number is sufficient to 
contain the required (720 + 48) samples, the frac- 
tional sample would mean that the structure was 
non-orthogonal on a line to line basis. This would 
cause significant design difficulties in the rest of 
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the standards converter, so the number of lines 
required was gradually increased, from 576, until a 
whole number of samples, in fact 770, existed in 
each line. 

The only format that achieves the orthogonal 
structure is the 585/60 format, which in addition 
gives a useful vertical blanking of four lines in the 
first field, five lines in the second field and fifty 
samples of horizontal blanking. 

In the 625/50 to 625/50 slow motion mode 
referred to below there is no requirement to store 
the active video of the 625 format within a 60 Hz 
period, so the interpolation and other processing is 
done in the normal 625/50 format. 

When converting from a low field rate to a 
higher rate the input time base corrector is re- 
quired to* produce a video stream at the output 
rate. This is done by occasionally repeating an 
input field. When the repeated field occurs, all the 
interpolator stores must be frozen so that the inter- 
polation is applied to the same input fields used to 
create the previous output field. 

If this technique were not used, two sets of 
interpolator and movement detector would be re- 
quired to make up the missing fieid. 

The above operation is performed when con- 
verting from 625/50 to 525/60. To allow 625 lines to 
exist during a 60-fields per second period again 
requires the 585/60 intermediate format to be 
adopted. During this process some of the interpo- 
lated lines will not be required, as only 525 have to 
be produced from the original 625. A time base 
converter is therefore required on the output to 
produce the final 525/60 format 

The amount of interpolation required is deter- 
mined by comparing input and output synchroniza- 
tion pulse phases. 

As mentioned above, motion analysis is per- 
formed on the luminance of the input video. The 
method employed involves a number of stages to 
arrive at a single motion vector for each pixel. 
Movement can be detected in the range t24 pixels 
horizontally and ±6 (field rate) vertically. 

In a first stage, motion in the picture at points 
on the screen spaced sixteen samples horizontally 
and eight lines vertically is determined using a 
block matching technique. The original motion vec- 
tors in a field are calculated every sixteenth sample 
and every eighth line. Each one of these points is 
at the centre of a search block. Conceptually each 
block is scanned ±24 samples horizontally, and + 8 
and -8 samples vertically over the next field each 
time generating the summation of the differences 
between the two fields over the area of the search 
block. The minimum overall difference then in- 
dicates in which direction the object at that point 
has moved. 



In practice, the above technique is applied in 
separate steps which greatly reduces the amount 
and complexity of hardware required: 

5 

Step 1. 

Test for minimum difference in just three posi- 
tions, centre position, sixteen samples to the left, 
io sixteen samples to the right. 

Step 2. Starting from point indicated above. 

75 Test for minimum difference in nine positions 
symmetrically distributed about the above starting 
point in steps of eight samples or lines. 



20 Step 3. Starting from point indicated above. 

Test for minimum difference in nine positions 
symmetrically distributed about the above starting 
point in steps of four samples or lines. 

25 

Step 4. Starting from point indicated above. 

Test for minimum difference in nine positions 
30 symmetrically distributed about the above starting 
point in steps of two samples or lines. 



Step 5. Starting from point indicated above 

35 

Test for minimum difference in nine positions 
symmetrically distributed about the above starting 
point in steps of one sample or line. 

40 

Step 6. 

After step 5. the motion of the object has been 
detected to the nearest pixel. A more accurate 

45 vector value can be achieved by adding a sixth 
step in which the difference produced at the final 
position indicated by step 5 is compared with the 
two differences above and below to adjust the 
vertical vector value and with the two differences to 

so the left and right to adjust the horizontal vector 
value. 

The above technique relies on achieving cor- 
relation between the reference search block and a 
similar block of video data on the following field 
55 (the search positions). In step 5 it is possible the 
true movement was a half pixel more or less than 
detected, but it is necessary for the best correlation 
to occur at this point, even although exact correla- 
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tion cannot be achieved. To ensure this occurs, the 
picture can be filtered both vertically and horizon- 
tally by a gaussian filter which has +6 dB attenu- 
ation at 1/2 Nyquist frequency. 

Similarly, for step 4, the picture can be filtered 
with a 6 dB attenuation at 1/4 Nyquist frequency, 
which allows a one pixel error in detection. 

Step 3 uses a picture filtered with a 6 dB 
attenuation at 1/8 Nyquist frequency allowing a two 
pixel error. 

Step 2 uses a picture filtered with a 6 dB 
attenuation at 1/16 Nyquist frequency allowing a 
four pixel error. 

Finally, step 1 uses a picture filtered with 6 dB 
attenuation at 1/32 Nyquist frequency allowing an 
eight pixel error. In addition, because the pictures 
are so heavily filtered during steps 1, 2, 3 and 4 t 
the samples can be reduced, for example halved in 
number, which still further greatly reduces the 
number of calculations and amount of hardware 
required. 

The effective search block size is sixteen lines 
high and forty-eight samples long. A large search 
block is necessary to detect accurately the move- 
ment of large plain areas. The central part of plain 
areas are unimportant as the values of the pixels at 
these points do not change from one field to the 
next, but the edges of such objects are obviously 
important. If the detection of motion is limited to 
±24 samples horizontally and ±8 lines verticaJly 
then a block of the above size would be the mini- 
mum size to ensure accurate motion detection. 

In the standards converters, depending upon 
the conversion modes, the luminance video enter- 
ing the motion analyzer 2 is in various forms of 
585-lines/60-fieIds per second. This might com- 
prise repeated lines for 525 input or repeated fields 
for 625 input. In addition, the input contains both 
field polarities. The first process is to ensure a 
continuity of data and single field polarity for the 
motion estimation processing. This is done by in- 
terpolation on the input data by a vector interface 
to maintain continuity, and filtration horizontally to 
aid subsequent motion detection/correlation. 

Separate outputs from this circuit are passed to 
motion estimation vector filters and motion detec- 
tion field stores/vector selectors. The output of the 
vector interface is, as described above, spatially 
continuous, single field polarity data The output to 
the field stores/vector selectors depends upon the 
input and output modes. In some modes it is 
continuous, and in others it contains repeated 
lines/fields. The vector filters and vector calculators 
perform the steps outlined above. 

The processing of the various steps is per- 
formed by vector calculators and a vector proces- 
sor. The vector calculators perform steps .1 to 5 
and the vector processor performs step 6. In addi- 



tion, the vector processor performs the second 
stage in the motion estimation, as follows: 

For each 8 x T6 block a choice is made of four 
from seven motion vectors, the seven motion vec- 

5 tors being the one for that particular block and the 
six for the six nearest blocks respectively. 

In addition, the vector processor also deter- 
mines the four most common motion vectors 
throughout the whole input field, these being called 

io modal motion vectors. The primary use of the 
modal motion vectors is in the border areas close 
to the edge of a field where it is not possible 
actually to calculate any local motion vectors. Also, 
if any one or more of the local motion vectors are 

is equal, then these are substituted for by the modal 
motion vectors. 

In the next stage of motion detection, for each 
pixel, the four motion vectors are tested by produc- 
ing the difference between the extrapolated posi- 

20 tions on field 0 to field 1 . During standards conver- 
sion a field needs to be interpolated between two 
fields; say between field 0 and field 1. So the 
motion vectors generated between these two fields 
are considered to, be most representative of the 
. 25 motion. Four motion vectors are used from these 
two fields. To decide which is the correct motion 
vector a pixel from field 0 is compared with a pixel 
from field 1 using the motion vector to decide 
where the pixel to be generated had come from on 

30 field 0 and where it has gone to by field 1 . Math- 
ematically, if the position x, y, z must be gen- 
erated, where; x = horizontal position, y = vertical 
position, z = temporal position between field 0 and 
field 1, the pixels used for comparison are as 

35 shown below. Field 0 is assumed to be at z = 0 and 
field 1 atz = 1. 
Pixel from field 0 
x° = x - (V h *z) 

y° = y- (v v *z) 

40 Pixel from field t 
x« = x + (1-z)V h 
y' = y + (1-z)V v 

V h = horizontal component of vector 
V v = vertical component of vector 

45 For each motion vector a modulus of the dif- 
ference between the pixels indicated in field 0 and 
field 1 is found. The minimum difference is as- 
sumed, as a first estimate, to indicate the correct 
motion vector. If a number of motion vectors pro- 

so duce a very similar difference then these motion 
vectors are tested again using a comparison be- 
tween fields -1 and 0. 
Pixels from field -1 
x- f = x-(1+z)V h 

55 y" 1 = y - (1 +z)V v 

The minimum modulus of difference of the remain- 
ing motion vectors produced by this second test is 
then considered to represent most accurately the 
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motion vector. 

If a number of motion vectors again have simi- 
lar differences then an option exists to assume no 
movement. If only the horizontal component varied 
and the vertical component did not, then only the 
horizontal component would be set to zero and the 
vertical component would be maintained at the 
detected value. If only the vertical component var- 
ied, then the horizontal component would be main- 
tained and only the vertical component set to zero. 
If the pixel difference chosen is too large then an 
option exists to set the whole motion vector to zero 
in both directions. 

A final stage is applied once every pixel has 
been assigned a motion vector. Here the motion of 
each pixel is tracked from one field to the next and 
a recursive filter applied to the vector value. This 
removes the effects of noise and small movement 
estimation errors and also' smooths the trajectory of 
the motion vectors. 

There are two possible ways of tracking the 
motion of a pixel. 

In the first, the motion vector for a pixel in field 
t is used to point to a pixel in field (t + 1). The 
motion vector determined for this pixel in field 
(t+1) is then recursively filtered to form the final 
motion vector for the pixel in field (t + 1 ). 

In the second, the motion vector for a given 
pixel in field t is used to point to a pixel in field (t- 
1 ). The motion vector from this pixel is then recur- 
sively filtered with the motion vector for the given 
pixel to form the final motion vector for this given 
pixel in field t. 

In either case the final output is a motion 
vector for each pixel which is passed from the 
motion analyzer 2 to the interpolator 1 to be em- 
ployed in aligning the four fields used in the stan- 
dards conversion process. 

The first standards converter for converting an 
input digital 625-line 50-fieIds per second television 
signal to an output digital 525-line 60-fields per 
second television signal is shown in detailed block 
form in Figure 2. 

The incoming video at 50-fields per second 
and a sample rate of 13.5 MHz, that is CCIR 601 
data, is supplied to a demultiplexer 31 which sepa- 
rates it into luminance components Y, synchroniz- 
ing signals SYNC and chrominance components 
UV. The luminance components Y are supplied to 
a 4-field luminance time-base corrector (TBC) 11 Y 
and the chrominance components UV are supplied 
to a 4-field chrominance TBC 1 1 C. The synchroniz- 
ing signals SYNC are supplied to a control 32 
together with an input field polarity signal from an 
external input, and an output field synchronizing 
reference signal from another externa] input The 
TBCs 11Y and 11C occasionally repeat fields, so 
that the output is at 60-fields per second. The 



control to the TBCs 1Y and 11C which causes the 
repetition of a field is derived from the input field 
synchronization pulses, and the required output 
field synchronization pulses. The comparison of the 

5 synchronization pulses also provides a temporal 
offset figure which indicates the amount of tempo- 
ral interpolation required at the outputs of the TBCs 
11 Y and 11C such that smooth motion at 60-fields 
per second would be observed. 

w When converting from 50-fields per second to 

60-fields in this way a line conversion of 625 to 525 
is necessary. It is therefore necessary to maintain 
the original 625 lines of information at a 60-fields 
per second rate so that they are all available to 

is form the interpolated lines. 

The standards converter uses an intermediate 
standard which is capable of containing all the 
active vertical information of the 50-fields per sec- 
ond signal at the 60-fields per second rate. The 

20 intermediate standard also contains all the active 
line information arranged orthogonally line by line 
still using the original 13.5 MHz sample rate. 

The intermediate standard use, and which is as 
explained above capable of meeting all these re- 

25 quirements, is a 585-line format at 60-fields per 
second. When sampled at 13.5 MHz each line of 
this format has exactly 770 samples. It is clear 
therefore that 585 lines is sufficient to contain the 
576 active lines of the 625-line format at a 60-fields 

30 per second rate. As the active line width is only 
720 samples there is still fifty samples of horizontal 
blanking. 

The luminance data (D) from the luminance 
TBC 11Y is supplied by way of a processing com- 

35 pensating delay 17Y to a luminance temporal shift 
register 16Y comprising four field stores (FS) 12Y, 
13Y. 14Y and 15Y. The luminance TBC 11 Y also 
supplies a -temporal freeze signal (F) by way of the 
delay 17Y to the shift register 16Y. The chromin- 

40 ance TBC 11C supplies the chrominance data (D) 
by way of a processing compensating delay 17C to 
a chrominance temporal shift register 16C which 
comprises four field stores 12C, 13C, 14C and 
15C. The chrominance TBC 11C also supplies a 

45 temporal freeze signal by way of the delay 17C to 
the shift register 16C. Associated with the shift 
register 16Y is a luminance interpolator 1Y which 
receives inputs from each of the field stores 12Y, 
1 3Y, 1 4Y and 1 5Y, and derives the 585-line format. 

so The output of the luminance interpolator 1Y is 
supplied to a 2-field luminance TBC 18Y. Asso- 
ciated with the shift register 16C is a chrominance 
interpolator 1C which receives inputs from each of 
the field stores 12C, 13C, 14C and 15C, and also 

55 derives the 585-line format. The output of the 
chrominance interpolator 1 C is supplied to a 2-field 
chrominance TBC 18C. When the outputs of the 
TBCs 11 Y and 11C are frozen, during a repeat 
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field, the shift registers 16Y and 16C are also 
frozen, so that four distinct consecutive fields of the 
input always exist in the shift registers 16Y and 
16C. Thus the shift registers 16Y and 16C are used 
to provide the ternporaJ taps for the interpolators 
tY and 1C. The present invention is particularly 
concerned with the interpolators 1Y and 1C. 

Each temporal tap produces four line taps at a 
position depending on the motion vectors, so that a 
two-dimensional filter can be used to provide the 
necessary interpolation. The interpolated picture 
will contain 576 active lines, so that a correct 
picture will be obtained when every sixth line in 
one field is dropped. The 484 lines left produce the 
active picture portion of the 525-line format. To 
enable lines to be dropped in this way, the outputs 
from the interpolators 1Y and 1C are supplied to 
the 2-field TBC 18. The TBCs 18Y and 18C write in 
all 576/2 lines, but only read out the required 484/2 
lines to provide the required output television sig- 
nal. The outputs of the luminance TBC 18Y and of 
the chrominance TBC 18C are supplied to a mul- 
tiplexer 34 which multiplexes the luminance com- 
ponents Y and the chrominance components UV to 
provide output CCIR 601 data in the form of a 
digital 525-line 60-fieids per second television sig- 
nal. 

The control 32 supplies control signals (C) to 
the luminance TBC 11 Y and to the chrominance 
TBC 11C. The control 32 also supplies control 
signals to the luminance TBC 18Y and the 
chrominance TBC 18C. It also supplies interpola- 
tion control signals (IC) to the luminance interpola- 
tor 1L and the chrominance interpolator 1C. 

The luminance data only, as supplied by the 
luminance TBC 11Y, is also supplied to the motion 
analyzer 2 shown in the upper part of Figure 2, so 
that motion vectors can be generated. In fact a 
frame delay is necessary between the TBCs 11 Y 
and 11C and the shift registers 16Y and 16C to 
allow for the time taken to process the motion 
vectors. The freezing of the shift registers 16Y and 
16C must therefore also be delayed by one frame, 
and these delays are provided by the delays 17Y 
and 17C. 

The motion analyzer 2 comprises a vector in- 
terface 35 to which the luminance data from the 
luminance TBC 11 Y is supplied, together with the 
interpolation control signal from the control 32. The 
vector interface 35 supplies data interpolated to 
625 lines to a vector filter 36 and a vector calcula- 
tor 37 which together perform the motion estima- 
tion described above. The output of the vector 
calculator 37 is supplied to a modal motion vector 
processor 38 and also to a sub-pixel motion es- 
timator. The motion vector processor 38 supplies 
four outputs and the sub-pixel motion estimator one 
output to a motion vector reducer 40. which sup- 



plies four outputs to a vector selector 41, with 
which the present invention is particularly con- 
cerned. 

The vector interface 35 also supplies the data 
5 interpolated to even fields to a processing com- 
pensating delay 42 to which it also supplies the 
received interpolation control signal, and also a 
temporal freeze signal (F) generated at the vector 
interface 35. The data from the delay 42 is sup- 
ra plied to a temporal shift register 43 which com- 
prises three field stores 44, 45 and 46 which sup- 
ply respective data outputs to the vector selector 
41. The delay 42 supplies the interpolation control 
signal to the vector selector 41 which supplies the 
15 selected motion vector to a recursive motion vector 
filter 47 t the output of which is the motion vector 
data which is supplied to the luminance interpolator 
1Y and to the chrominance interpolator 1C. 

The way in which the motion analyzer 2 de- 
20 rives the motion vector data has been described in 
detail above, and will be further described below, 
but the operation of the elements 35 to 43 and 47 
will now be briefly described. 

The vector interface 35 receives the luminance 
25 data from the luminance TBC 11Y, and the inter- 
polation control signals from the control 32. It sup- 
plies 625-line data, normally contained within the 
585/60 format, to the vector filter 36. It also sup- 
plies data to the delay 42. These data must contain 
30 a picture which is the same line standard as the 
required output, again normally contained within 
the 585/60 format. Each field of the interpolated 
data is also made to appear even. 

The vector filter 36 produces the filtered pic- 
as ture data required for steps 1 to 5 above of the 
motion detection. The filtered picture data are sup- 
plied in sample reduced form to the vector calcula- 
tor 37. 

The vector calculator 37 operates on the fil- 

40 tered and sample-reduced data from the vector 
filter 36 using an algorithm described in the terms 
of the steps 1 to 5 above of the motion detection. 
The process is essentially a two-dimensional binary 
search for motion down to pixel/line resolution. For 

45 each field, 1200 motion vectors are generated and 
are supplied to both the modal motion vector pro- 
cessor 38 and to the sub-pixel motion estimator 39. 
It also supplies surrounding weighted absolute dif- 
ference (WAD) values as calculated by step 5 

50 above to the sub-pixel motion estimator 39. For 
details of WAD calculations, see 'Advances in Pic- 
ture Coding 1 , Musmann et al, Proceedings of the 
IEEE, April 1985. The specific WAD value which is 
the minimum in step 5 above of the motion detec- 

55 tion provided a figure of merit (FOM). 

The vector processor 38 calculates the four 
most common motion vectors that are detected in 
each field and supplies them to the vector reducer 



8 



15 



EP 0 294 957 A2 



16 



40. 

The sub-pixel motion estimator 39 receives the 
motion vectors from the vector calculator 37 to* 
gether with the surrounding WAD values. From 
these it estimates sub-pixel movement to be ap- 
pended to the motion vector values. With each 
motion vector its corresponding final WAD value is 
also supplied to the vector reducer 40. 

The vector reducer 40 receives the motion 
vectors from the vector processor 38 and from the 
sub-pixel motion estimator 39. For each motion 
vector from the sub-pixel motion estimator 39, the 
six motion vectors closest to it are grouped to- 
gether. For each motion vector there are then elev- 
en choices. The reduction process selects four 
motion vectors from the eleven for supply to the 
vector selector 41 . 

The vector reducer 40 supplies the vector se- 
lector 41 with four representative motion vectors for 
each sixteen pixel by eight line block of the picture. 
By comparing pixels over three fields, the vector 
selector 41 selects the single best motion vector 
for each pixel in the picture, as described in more 
detail below. The motion vector selected is sup- 
plied to the motion vector filter 47. 

The delay 42 delays the data by one frame 
less twenty-one lines to compensate for other de- 
lays in the system. 

The temporal shift register 43 holds and sup- 
plies the three fields of data used by the vector 
selector 41. 

The motion vector filter 47 tracks a motion 
vector from one field to another so applying some 
filtering to the motion vectors by combining motion 
vectors in different fields, so reducing motion de- 
tection errors. The output of the motion vector filter 
47 is supplied to the luminance and chrominance 
interpolators 1Y and 1C to control the alignment of 
the field data. 

Exactly the same hardware can be used as a 
slow motion processor with good motion portrayal 
for either a 625/50 or a 525/60 television signal. It 
is not however necessary to use the vertical inter- 
polator to provide the line number conversion. In all 
cases the control 32 determines what action is 
required by recognizing the input/output standard 
from the input and output field synchronization 
pulses. In slow motion the input field polarity is 
used. 

Whereas in 50-fields per second to 60-fields 
per second conversion a field, was occasionally 
repeated, in slow motion the field is repeated the 
same number of times as the input field is re- 
peated. As repeated fields are not written into the 
shift registers 16Y and 16C, the shift registers 16Y 
and 16C again contain distinct consecutive fields. 
Indeed if a video tape recorder reproduces without 
any interpolation of its own, the original interlace 



structure is maintained allowing full resolution pic- 
tures to be produced. The temporal offset required 
is calculated by comparing the actual field rate 
pulses, by they 50-fields per second or 60-fields 

5 per second, with the rate at which a new field is 
received. To determine the temporal offset in this 
way. the system needs a signal to be available 
which indicates the true field polarity of the field 
being repeatedly replayed. The vertical interpolator 

70 will always produce the field polarity required at the 
output. 

Conceptually the TBCs 1 1 Y and 11C are not 
really required for slow motion operation, but their 
presence does provide a frame synchronization 

is facility and also simplifies the system configuration. 
The second standards converter for converting 
an input digital 525-line 60-fields per second televi- 
sion signal to an output digital 625-line 50-fields 
per second television signal is shown in detailed 

20 block form in Figure 3. 

In this case, interpolation requires that all the 
input data is available in a consecutive form. In this 
case it would not therefore be possible to convert 
to 50-fields per second before the interpolators 1Y 

25 and 1C. The input data however contains only 484 
active lines and the interpolators 1Y and 1C must 
produce 576. The 2 field TBCs 18Y and 18C are 
therefore positioned at the front of the standards 
converter to provide the necessary time slots for 

30 484-line to 576-line conversion. 

The original continuous line structure is written 
into the TBCs 18Y and 18C but is read out in the 
585-line standard with approximately every sixth 
line being blank. The interpolators 1Y and 1C are 

35 then used to produce a continuous picture at the 
output line rate by freezing its line stores during 
the blank input line, and producing the required 
additional line at the output, so ensuring that a 
spatially correct picture is produced. The required 

40 temporal offset is detected and applied as in the 
first standards converter, although the interpolation 
is applied such that a field can occasionally be 
dropped leaving the motion smooth. The field is 
dropped such that 60-fields per second to 50-fields 

45 per second conversion is achieved. The dropping 
of a field is achieved by using the 4-field TBCs 1 1 Y 
and 11C at the output. 

Thus the second standards converter differs 
from the first standards converter shown in Figure 

so 2 in only minor respects. In particular, the lu- 
minance TBCs 11Y and 18Y are interchanged, and 
the chrominance TBCs 1 1C and 18C are also inter- 
changed. Also, no temporal freeze signals are re- 
quired. 

55 In both cases the control 32 has various func- 

tions as follows; controlling the reading and writing 
of the TBCs 11Y, 11C, 18Y and 18C; generating a 
temporal offset number, and in the case of the first 
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standards converter the temporal freeze signal, and 
generating a vertical offset number together with 
vertical interpolation control signals. These func- 
tions will now be described in more detail. 

Firstly, the 2-field luminance and chrominance 
TBCs 18Y and 18C always switch between field 
stores at the end of every 60 Hz field. However, the 
operation of the 4-fieid luminance and chrominance 
TBCs 11 Y and 11 C depend on the mode of opera- 
tion, and their control is also associated with the 
generation of the temporal offset signal. In fact the 
control of the luminance and chrominance TBCs 
11 Y and T1C is determined from the input and 
output field synchronizing signals. 

The derivative of the temporal offset signal in 
the case of 525/60 to 625/50 operation will now be 
described with reference to Figures 4 and 5. 

In Figure 4, the control 32 is shown as includ- 
ing a line counter 61, and first and second latches 
62 and 63. A line clock signal is supplied to a clock 
terminal of the line counter 61 , while the input field 
synchronizing signal is supplied to a reset terminal 
of the line counter 61 and to a clock terminal of the 
second latch 62. The output field synchronization 
signal is supplied to a clock terminal of the first 
latch 62. The output of the line counter 61 is 
supplied to the input of the first latch 62, the output 
of which is supplied to the input of the second 
latch 63, the output of which is the temporal offset 
signal supplied to the luminance and chrominance 
shift registers 11Y, 11C, 18Y and 18C. 

The input and output field synchronizing sig- 
nals are shown in Figures 5A and 5B respectively. 
Figure 5C shows the output of the line counter 61 
which repetitively counts from 0 to 524. Figures 5D 
and 5E show the outputs of the first and second 
latches 62 and 63 respectively. By latching the 
counter 61, the required proportion of the input 
field period is determined. The temporal shift value 
tn indicates the position between two input fields 
where the output field must be interpolated such 
that when the shaded field shown in Figure 5A is 
dropped, continuous motion still occurs. Thus, the 
field which uses the temporal offset shown shaded 
in Figure 5E is the one that is dropped. It will be 
seen by reference to Figures 5A and SB, that the 
field which is dropped is the one which does not 
have a new temporal shift associated with it. The 
field (arrowed) which is to be dropped is indicated 
to the following circuitry by the temporal freeze 
signal. 

The derivation of the temporal offset signal in 
the case of 625/50 to 525/60 operation will now be 
described with reference to Figures 6 and 7. 

In Figure 6, the control 32 is shown as includ- 
ing a line counter 71 and a latch 72. A line clock 
signal is supplied to a clock terminal of the line 
counter 71 , while the input field synchronizing sig- 



nal is supplied to a reset terminal of the line 
counter 71. The output field synchronization signal 
is supplied to a clock terminal of the latch 72. The 
output of the line counter 71 is supplied to the 
5 input of the latch 72, the output of which is the 
temporal offset signal supplied to the luminance 
and chrominance shift registers 11 Y, 11C, 18Y and 
18C. 

The input and output field synchronizing sig- 

70 nals are shown in Figures 7A and 7B respectively. 
Figure 7C shows the output of the line counter 71 
which repetitively counts from 0 to 624. Figure 7D 
shows the output of the latch 72. By latching the 
counter 71, the required proportion of the input 

75 field period is determined. Thus, the temporal shift 
value tn again indicates the position between two 
input fields where the output field must be interpo- 
lated, such that if the shaded field is repeated, 
continuous motion stilt occurs. The field which is 

20 repeated is the one which has two temporal shift 
values associated with it. The field (arrowed) which 
is to be repeated is indicated to the following 
circuitry by the temporal freeze signal. 

The deviation of the temporal offset signal in 

25 the case of slow motion whether at 525/60 to 
525/60 or 625/50 to 625/50 is the same, and will 
now be described with reference to Figures 8 and 
9. 

In figure 8, the control 32 is shown as including 
30 a line counter 81. a field counter 82. first to fourth 
latches 83 to 86, an exclusive-OR gate 87 and a 
scaler 88. The input field synchronizing signal is 
supplied to a clock terminal of the first latch 83, to 
a clock enable terminal of the field counter 82, and 
35 to a second reset terminal of the line counter 81. 
The input field polarity signal is supplied to the first 
latch 83 and thence to the second latch 84 and 
also to one input of the gate 87. The second latch 

84 supplies an output to the second input of the 
40 gate 87, the output of which is supplied to a first 

reset terminal of the line counter 81, to a reset 
terminal of the field counter 82 and to a clock 
terminal of the third latch 85, which forms a speed 
detector latch. A line clock signal is supplied to a 

45 clock terminal of the second latch 84, and to re- 
spective clock terminals of the tine counter 81 and 
the field counter 84. The output of the line counter 
81 is supplied to an input terminal of the scaler 88, 
and the output of the field counter 82 is supplied to 

so an input of the third latch 85 and also to an offset 
input terminal of the scaler 88. The output field 
synchronizing signal is supplied to a clock terminal 
of the fourth latch 86. The output of the third latch 

85 is supplied to a scale factor terminal of the 
55 scaler 88, the output of which is supplied to the 

fourth latch 86, the output of which is the temporal 
offset signal. 

The input field synchronizing signal and the 
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input field polarity signal are shown in figures 9A 
and 9B respectively. Figure 9C also indicates the 
input field synchronizing signals and Figure 9D the 
output field synchronizing signals. Figures 9E and 
9F indicate the operations of the field counter 82 
and the line counter 81, which are respectively 
counting fields and lines from 0 to N. Figure 9G 
indicates the output of the fourth latch 86 which is 
the temporal offset signal. Figure 9H indicates the 
temporal freeze signal (which is active when low), 
and, as indicated by the arrows, the shaded field 
that uses the temporal offset shown in a repeat of 
the previous field that used the temporal offset t1. 

To generate the temporal freeze signal, the 
control 32 is shown in Figure 10 as including a 
synchronous RS Hip-flop 91, a latch 92, an inverter 
93 and an AND-gate 94. The output field synchro- 
nizing signal is supplied to one input of the flip-flop 
91, to the input of the inverter 93 and to a clock 
enable terminal of the latch 92. The input field 
synchronizing signal is supplied to the other input 
of the flip-flop 91. while a line clock signal is 
supplied to clock terminals of the flip-flop 91 and 
the latch 92. The output of the flip-flop 91 is 
supplied to one input of the gate 94, which re- 
ceives at its other input the output of the inverter 
93. The output of the gate 94 is supplied to the 
input of the latch 92, the output of which forms the 
temporal freeze signal. The operation of this circuit 
is such that if more than one output field synchro- 
nizing pulse follows an input field synchronizing 
pulse, a freeze occurs. 

Referring back to Figure 2, the generation of 
the vertical offset number by the control 32 will 
now be described. The same address generator 
which reads data from the luminance TBC 11Y into 
the luminance interpolator 1Y and the motion ana- 
lyzer 2, also addresses an erasable programmable 
read-only memory (EPROM) which provides the 
vertical offset number together with vertical freeze 
signals when required. 

(In the Rgure 3 arrangement which is used for 
525/60 to 625/50, the read addresses of the lu- 
minance TBC 18Y are used, but in ail other modes 
the read addresses of the luminance TBC 11Y are 
used.) 

The vertical offset number is generated assum- 
ing that both the input and the output fields are 
even, and it then indicates the position between 
two input lines where the output line must be 
interpolated such that a non-distorted picture would 
be produced if: a line were occasionally dropped in 
625/50 to 525/60 conversion, or 
a line were occasionally repeated in 525/60 to 
625/50 conversion. 

When a line is repeated by the luminance TBC 
1 1 Y (18Y), a vertical freeze signal is generated. 

If the input fields are not both even, then the 



interpolators 1 Y and 1C must make use of the input 
field polarity and output field polarity to ensure 
correct interpolation. 

The contents of the EPROM are generated in a 
5 way similar to that described above in connection 
with Figure 10 for the temporal offset signal, using 
the known line position in both a 525 and a 625 
picture. 

The form and operation of the motion vector 
w filter 47 (Figure 2), with which the present invention 
is particularly concerned, will now be described in 
more detail with reference to Figures 11 to 21. 

As stated above, the vector reducer 40 sup- 
plies the vector selector 41 with four representative 

is vectors for each sixteen pixel by eight line block of 
the picture. By comparing pixels over three fields, 
the vector selector 41 selects the single best mo- 
tion vector for each pixel in the picture, and this 
motion vector is supplied to the motion vector filter 

20 47 where motion of each pixel is tracked from one 
field to the next and recursive filtering applied to 
the motion vector value. This removes the effects 
of noise and small movement estimation errors and 
also smooths the trajectory of the motion vectors. 

25 There are two possible ways of tracking the 

motion of a pixel. 

In the first (negative vector) mode,, the motion 
vector for a given pixel in field t is used to point to 
a pixel in field (t-1). The motion vector from this 

30 pixel is then recursively filtered with the motion 
vector for the given pixel to form the final motion 
vector for this given pixel in field t. 

In the second (positive vector) mode, the mo- 
tion vector for a pixel in field t is used to point to a 

35 pixel in field (t+l). The motion vector determined 
for this pixel in field (t+1) is then recursively 
filtered with the motion vector for the given pixel to 
form the final motion vector for this given pixel in 
field (t+1). 

40 This recursive filtering and associated process- 

ing will now be considered in more detail with 
reference to Figures 11 to 21. Basically the inten- 
tion is that the filtered motion vectors used in the 
interpolation should be smooth and predictable in 

45 nature, so improving the overall subjective effect in 
the final picture. -In this context the term 'recursive 
filtering and associated processing 1 actually en- 
compasses two separate processes. The first is the 
actual 'recursive filtering 1 and the second could be 

so termed 'majority vector determination 1 , which is 
primarily concerned with the detection of motion 
vectors which are substantially different, for exam- 
ple as a result of a scene change. It should be 
noted that first and second does not necessarily 

55 indicate the order in which these processes take 
place and that the second process can be omitted 
in some embodiments. 

Consider the motion of a single object across a 
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static background. When photographed and the 
resulting video passed to the standards converter, 
motion vectors are generated showing the trajec- 
tory of the object. Due to the video noise and small 
errors in determining the motion vectors it is un- 
likely that the motion vectors will be perfectly con- 
tinuous. However, the motion of the object should 
be readily discernible as indicated in Figures 11 A 
and 11 B where Figure 11A indicates the motion of 
an object in sequential fields, and Figure 11 B in- 
dicates successive motion , vectors generated from 
sequential incoming fields. 

By comparing tfre motion vectors from sequen- 
tial fields, some erroneous conditions and some 
picture content information can be determined. For 
example, suppose the motion vectors from three 
sequential fields are derived as shown in Figure 12 
by field stores 10t and 102, and are supplied to a 
comparator 1Q3. The resulting comparisons are: 

(a) In a first case (Figure 1 3): 

(V,. 2 is similar to V t . T ) are different from V t 
Vt is so significantly different from V t . 2 and that 
it seems likely that the motion of the object stop- 
ped at a point A (for example, as the result of a 
scene change). V, should not therefore be consid- 
ered as a continuation of V M and V t . 2 . 

(b) in a second case (Figure 14): 

(V t _2 is similar to V t ) are different from V M 
It appears likely that V t . 2 and V t are part of a 
continuous motion. Therefore, it is probable that 
there has been an error in calculating V M and it 
should therefore be disregarded. 

(c) In a third case (Figure 1 5): 
^i ^t^i s ^sM , •• n i l, i 
The motion is continuous and it would be reason- 
able to assume that V t . 2 , V M and V t represent the 
trajectory of the object 

It is apparent that in determining the equality, 
or inequality of the above vectors, some degree of 
flexibility must be included. It is unlikely that two 
sequential vectors will be identical but within de- 
fined limits of accuracy they might still represent 
continuous motion. The purpose of recursively fil- 
tering sequential motion vectors is to improve the 
motion portrayal of an object which might otherwise 
contain small errors. These small errors can arise 
as the result of noise in the incoming video affect- 
ing the generation of motion vectors and also as a 
result of small errors in the actual determination of 
these vectors. Figure 16 shows an example of the 
effect of recursive filtering in producing a resultant 
motion vector. 

The recursive filtering is done in an infinite 
impulse response (MR) filter generally as shown in 
Figure 17. The IIR filter comprises a multiplier 111 
to which the input is supplied, an adder 112 from 
which the resultant motion vector is derived, and a 
field store 113 and a multiplier 114 forming a loop 



with the adder 112. Appropriate coefficients are 
supplied to the multipliers 1lt and 114. 

There are two possible implementations. They 
differ in respect of obtaining V x from the field store 

5 113. The first implementation is termed negative 

vector and the second is termed positive vector. '* 

In the negative vector mode, consider the in- 
coming motion vector V, in Figure 18. The motion 
vector with which V t is combined (V x ) must be 

70 obtained from the field store. In the negative vector 
mode this is done by using a negative version of V t 
to point to the position in the previous field from 
which it came. Ideally, V x should be the resultant 
motion vector to data of V M , V t . 2( etc. However, it 

75 is unlikely that -(V t ) will point exactly to a resultant 
motion vector. Instead it will point to a resultant 
motion vector in the same vicinity as the required 
resultant motion vector. Generally the difference 
between the pointed to and required motion vector 

20 wili be minimal and lead to a satisfactory output. 

The modified IIR filter for the negative vector 
mode is shown in Figure 19. As compared with the 
basic filter of Rgure 17, the input motion vector is 
additionally supplied to a negator 115, and the 

25 resulting negated input motion vector is supplied to 
an adder 116. A read address generator supplies 
basic read addresses for the field store 113 to the 
adder 116, while a write address generator 118 
supplies write addresses directly to the field store 

30 113. 

The input motion vector for each pixel is repre- 
sented by x,y coordinates, each of which is an 8-bit 
number. Therefore the bus width throughout the 
motion vector filter of Figure 19 is sixteen bits. 

cycle, and is passed to both a four clock cycle 
delay 119 and the negator 115. The purpose of the 
four clock cycle delay will be mentioned below. 
From the delay 119, the motion vector is passed to 

40 the 'input 1 multiplier 111 where it is multiplied by a 
coefficient 1. The coefficient 1 is continuously sup- 
plied to the multiplier 111 from a PROM 120. The 
value of the coefficient 1 is selectable under the 
control of an input 'coefficient select 1\ This, to- 

45 gether with an input 'coefficient select 2 t which is 
supplied to a PROM 121 that continuously supplies 
a coefficient 2 to the multiplier 114, allows the • 
response to the filter to be actively changed, for 
example as the result of detecting the condition (a) ^ 

so in the majority vector determination. 

The 16-bit output of the multiplier 111 is sup- 
plied to the adder 112 to be added to the output of 
the 'feedback 1 multiplier 114, that is the product of 
coefficient 2 and the output of the field store 113. 

55 The output of the adder 112 is the required 
smoothed output motion vector , and forms the 
output of the filter, as well as being supplied to the 
input of the field store 113. 

12 
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The field store 113 comprises sufficient mem- 
ory to store a field of motion vectors, which in the 
present case is 864 per tine for each of 292.5 lines 
per field, a total of 252720 times sixteen bits. As 
motion vectors enter the field store 113 they are 
written into sequential locations addressed by the 
write address generator 118, which may simply be 
a counter chain clocked by the system clock, and 
reset at the beginning of each field by a field 
strobe pulse. 

In this negative vector mode, the motion vector 
read from the field store 1 13 is obtained by using a 
negated version of the incoming motion vector to 
point to the approximate location of the resultant 
vector. This is done by first generating a sequential 
basic read address corresponding to the location in 
the field of the required motion vector if the value 
of the input motion vector were zero. To this is 
added the negated value of the actual input vector, 
so as to point to the iocation of the feedback 
vector. Using this address, the field store 113 is 
read and the feedback motion vector is supplied to 
the multiplier 114. 

in the multiplier 114, the feedback motion vec- 
tor is multiplied by coefficient 2, and the result of 
this multiplication is supplied to the adder 112. 

The four clock cycle delay of the input motion 
vector is required to delay it from being passed to 
the multiplier 113 while the same motion vector is 
used to read a feedback motion vector from the 
field store 113. In other words, it takes four clock 
periods to negate the input motion vector, to add it 
to the basic read address, and to use the resulting 
read address to read the field store 113. 

In the positive vector mode, consider the in- 
coming motion vector V t in Figure 20. This time, 
instead of affecting the read address of the field 
store 113, V t affects the write address. Once the 
resultant motion vector from the combination of V t 
and V x has been determined, it is stored in the field 
store 113 at the location pointed to by the resultant 
motion vector. 

The modified II R filter for the positive vector 
mode is shown in Figure 21 where the same refer- 
ence numerals are used for elements correspond- 
ing to those in Figure 19, even where the precise 
arrangement thereof is slightly different In particu- 
lar, it will be noted that the output of the adder 112 
is supplied to the field store 113 by way of a delay 
131 which is provided to compensate for the delay 
in a further adder 132. to which the output of the 
adder 112 is also supplied, and which also receives 
the basic write addresses from the write address 
generator 118. Finally, the output of the field store 
113 is supplied to a concealment device 113 to 
which it also supplies concealment flags. 

In the positive vector mode, as each input 
motion vector enters the filter, it is passed the 



'input' multiplier 111 and multiplied by coefficient 1 
as before. The result is then supplied to the adder 
1 12 and added to the result of the feedback motion 
vector which has been multiplied by coefficient 2 in 
5 the feedback* multiplier 114. As before, the output 
of the adder 112 is the final smoothed motion 
vector which forms the output of the filter and is 
also supplied by way of the delay 131 to the field 
store 113 and directly to the adder 132. 

io Instead of being sequentially written in the field 
store 113, as in the negative vector mode, the 
motion vector is written into the location in the field 
to which it is pointing. That is. if the smooth motion 
vector is on line 30, for example, at sample posi- 

i$ tion 50, and has a value +8, +2, then it will be 
written to location 30 + 8, 50-2. 

This is done by first generating the basic write 
address, that is the address of the location where 
the motion vector would be written if the value of 

20 the motion vector were zero, for example 30, 50. 
The value of the motion vector, for example 8, 2, is 
then added to the basic write address to generate 
the actual write address 38, 48. This determination 
of the actual write address takes a few clock cy- 

25 cles, and therefore the motion vector is delayed 
accordingly at the data input to the field store 113 
by the delay 131. 

One field later the motion vector is read from 
the field store 113. This is done sequentially by the 

30 read addresses supplied by the read address gen- 
erator 117, which again can be counter chain cloc- 
ked by the system clock and reset by a field strobe 
pulse. 

Subsequent to the field store 113 is the con- 

35 cealment device 133 which operates to conceal 
any locations which were not written in during the 
writing of the preceding field. The locations which 
require concealment are identified by a conceal- 
ment flag which is passed, together with the motion 

40 vector, from the field store 113. The concealment 
flag is determined as follows. 

During the reading of a field of motion vectors, 
the flag for each location is reset low as each 
location is read. Then, during the writing of the 

45 motion vectors, when a given location is written in, 
the respective flag is set high. Once a field of 
motion vectors has been written, any flags which 
remain low indicate a location which was not writ- 
ten in, and therefore requires concealing. 

so Two possible methods of concealment will now 

be briefly mentioned, either of which is selectable 
by a control signal indicated in Figure 21 as Pos- 
Zero. The first method is simply to insert a motion 
vector with a value of zero. This will have the effect 

55 of causing the output of the filter to tend towards 
zero. However, the output will only actually begin to 
approach zero if a significant number of motion 
vectors on consecutive fields are concealed. 
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The second method is to replace with a spa- 
tially neighbouring motion vector. In the case where 
the concealed motion vector is one of a group of 
identical or substantially identical motion vectors, 
this will lead to better results. However, if it is not, 5 
the result of replacing with a wrong value can be 
worse than replacing with a value of zero. Because 
of this, and also if a series of motion vectors are to 
be replaced, that is, for example, a 'good* motion 
vector A is used to replace a 'bad' motion vector B, io 
which is then used to replace a 'bad' motion vector 
C, account is kept of how many times a single 
'good 1 motion vector is used to replace consecutive 
'bad* motion vectors. Upon reaching a preset 
count, which may, for example, be in the range one 75 
to sixteen, concealment reverts to inserting zeros. 
After the concealment, the feedback motion vector 
is multiplied by coefficient 2 and supplied to the 
adder 112. 

In practice, the field store 113 whether in the 20 
positive or negative vector mode may be divided 
into two parts. This is because it is necessary to 
have random access to the field store 113, this 
being required for reading in the negative vector 
mode and for writing in the positive vector mode. 25 
Static random access memory (SRAM) devices 
therefore commend themselves, but a complete 
field store of SRAM devices consumes a substan- 
tial amount of power and board space. Dynamic 
random access memory (DRAM) devices on the 30 
other hand need to be multiplexed together due to 
their relatively slow access time, so preventing 
random access. One possibility, therefore, is to use 
a combination of both SRAM and DRAM devices. ^ 
This is possible because it is only necessary to 35 
have random access to a limited number of loca- 
tions at any one time. 

The maximum motion vector values are plus 
and minus sixteen vertically and plus and minus 
twenty-four horizontally. Therefore if thirty-two lines 40 
of the field are stored in SRAM devices, then 
random reading and writing can be performed. The 
rest of the field store 113 can be formed of DRAM 
devices to save power and board space. In the 
positive vector mode the 32-lines SRAM device 45 
store is positioned ahead of the remainder of the 
field store, that is one field minus thirty-two lines, 
formed of DRAM devices. Hence, it is possible to 
write randomly in the SRAM devices. One the other 
hand, in the negative vector mode, the 32-lines so 
SRAM device store is positioned after the remain- 
der of the field store, that is one field minus thirty- 
two lines, formed of DRAM devices. 

Attention is drawn to seven other European 
patent applications, corresponding to UK patent 55 
applications 8728445, 8728446. 8728448, 8728449, 
8728450, 8728451 and 8728452 relating to similar 



subject matter, which we filed on the same day as 
the present application, and the disclosures in 
which are incorporated herein by reference. 

Claims 

1 . A television standards converter comprising: 
means (36 to 40) for analyzing the motion between 
consecutive fields of an input television signal of 
one television standard and for deriving motion 
vectors in dependence on said motion; 
characterized by; 

means (47) recursively to filter said motion vectors; 
means (1) then to align said fields in dependence 
on said filtered motion vectors so as effectively to 
represent static pictures; and 
means (1) to effect conversion using said static 
pictures to derive the required output television 
signal of a different television standard. 

2. A television standards converter according to 
claim 1 wherein said recursive filter means (47) 
operates such that the motion vector for a given 
pixel in a given field is used to point to a pixel in 
the preceding field, and the motion vector for said 
pixel in said preceding field is then recursively 
filtered with the motion vector for said given pixel 
to form the final motion vector for said given pixel 
in said given field. 

3. A television standards converter according to 
claim 1 wherein said recursive filter means (47) 
operates such the motion vector for a given pixel in 
a given field is used to point to a pixel in the 

. , " j t "i^Vj^S^^Sp ^?ftr^j~^!"^ * and the mp.tion vector determined 
for said pixel in said succeeding field is then recur- 
sively filtered with the motion vector for said given 
pixel to form the final motion vector for said pixel in 
said given field. 

4. A television standards converter according to 
claim 1, claim 2 or claim 3 wherein said motion 
vectors are derived by determining motion in said 
image at points spaced a predetermined number of 
samples horizonally and a predetermined number 
of samples vertically by a block matching tech- 
nique. 

5. A television standards converter according to 
claim 4 wherein said block matching technique 
comprises the steps of: 

testing for minimum difference in three positions, 
the centre position of a block, a predetermined 
number of samples to the left, and the same pre- 
determined number of samples to the right 
starting from the point indicated above, testing for 
minimum difference in nine positions symmetrically 
distributed about the above starting point in steps 
of a smaller predetermined number of samples or 
lines; 

starting from the point indicated above, testing for 
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minimum difference in nine positions symmetrically 
distributed about the above starting point in steps 
of a still smaller predetermined number of samples 
or lines; 

starting from the point indicated above, testing for 5 
minimum difference in nine positions symmetrically 
distributed about the above starting point in steps 
of a still smaller predetermined number of samples 
or lines; and 

starting from the point indicated above, testing for io 
minimum difference in nine positions symmetrically 
distributed about the above starting point in steps 
of one sample or line. 

6. A television standards converter according to 
claim 5 where said block matching technique com- 75 
prises a further step, subsequent to the last step of 
claim 5, of comparing the difference produced at 

the final position indicated by said last step with 
the two differences above and below to adjust the 
vertical vector value, and with the two differences 20 
to the left and right to adjust the horizontal vector 
value. 

7. A 625-tine 50-fields per second to 525-line 
60-fields per second television standards converter 
comprising: 25 
a first time base corrector (11) for receiving an 
input 625-iine 50-fields per second digital television 
signal; 

characterized by: 

a motion analyzer (36 to 40) connected to the 30 
output of said first time base corrector (11) for 
analyzing motion in said input television signal and 
for deriving motion vectors in dependence on said 
motion; means (47) recursively to filter said motion 
vectors; 35 
a shift register (12) also connected to the output of 
said first time base corrector (1 1 ); 
an interpolator (1) for deriving samples of a re- 
quired output 525-line 60-fields per second digital 
television signal in dependence on samples de- 40 
rived from said shift register (12) and said filtered 
motion vectors; and 

a second time base corrector (18) for assembling 
said derived samples to form said output television 
signal. 45 

8. A television standards converter according to 
claim 7 wherein said first time base corrector (1 1 ) 
is a 4-field time base corrector (11) and said sec- 
ond time base corrector (18) is a 2-field time base 
corrector (18). so 

9. A television standards converter according to 
claim 8 wherein said 4-field time base corrector 
(11) derives 585-line 60 fields per second television 
signal from said input television signal for supply to 

said shift register (12). 55 

10. A 525-line 60-fields per second 625-line 50- 
fields per second television standards converter 
comprising: 



a first time base converter (18) for receiving an 
input 525-line 60-fields per second digital television 
signal; 

characterized by: 

a motion analyzer (36 to 40) connected to the 
output of said first time base corrector (18) for 
analyzing motion in said input television signal and 
for deriving motion vectors in dependence on said 
motion vectors; 

means (47) recursively to filter said motion vectors; 
a shift register (12) also connected to the output of 
said first time base corrector (18); 
an interpolator (1) for deriving samples of a re- 
quired output 625-line 50-fields per second digital 
television signal in dependence . on samples de- 
rived from said shift register (12) and said filtered 
motion vectors; and 

a second time base corrector (11) for assembling 
said derived samples to form said output television 
signal. 

11. A television standards converter according 
to claim 10 wherein said first time base corrector 
(18) is a 2-field time base corrector (18) and said 
second time base corrector (11) is a 4-field time 
base corrector (11). 

12. A television standards converter according 
to claim 1 1 wherein said 2-field time base corrector 
(18) derives a 585-line 60-fields per second televi- 
sion signal from said input television signal for 
supply to said shift register (12). 

13. A slow motion processor comprising: 

an input circuit (31, 11) for receiving an input digital 
television signal; 

a motion analyzer (36 to 40) for analyzing motion in 
said input digital television signal and for deriving 
motion vectors in dependence on said motion; 
characterized by: 

means (47) -recursively to filter said motion vectors; 
a shift register (12) for holding successive different 
fields of said input television signal; 
an interpolator (1) for deriving samples of a re- 
quired slow motion output digital television signal in 
dependence on the degree of slow motion, sam- 
ples derived from said shift register (12), and said 
filtered motion vectors; and 

a 2-field time base corrector (18) for assembling 
said derived samples to form said slow motion 
output television signal. 

14. A slow motion processor according to claim 

13 wherein said input circuit (31, 11) comprises is 
a 4-field time base corrector (1 1). 

15. A slow motion processor according to claim 

14 wherein said 4-field time base corrector (11) 
derives a 585-line 60-fields per second television 
signal from said input television signal for supply to 
said shift register (12). 
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© A television standards converter comprises a 
motion analyzer (36 to 40) for analyzing the motion 
between consecutive fields of an input television 
signal of one television standard and for deriving 
motion vectors in dependence on the motion, a filter 
(47) for recursively filtering the motion vectors, and 



an interpolator (1) for aligning the fields in depen- 
dence on the filtered motion vectors so as effectively 
to represent static pictures, and means (1) to effect 
conversion using the static pictures to derive the 
required output television signal of a different televi- 
sion standard. 
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